Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abartlet tombstone reanimation wip #3

Open
wants to merge 97 commits into
base: master
Choose a base branch
from

Conversation

kamenim
Copy link
Owner

@kamenim kamenim commented Nov 7, 2014

No description provided.

@kamenim kamenim force-pushed the abartlet-tombstone-reanimation-wip branch 3 times, most recently from 247bda9 to a9c3c8b Compare November 20, 2014 03:11
@kamenim kamenim force-pushed the abartlet-tombstone-reanimation-wip branch 2 times, most recently from 25be16e to f4fa20b Compare November 26, 2014 06:15
vlendec and others added 25 commits December 8, 2014 00:33
We don't really need a talloc context here

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
vfswrap_fallocate() is broken in that it can call posix_fallocate()
which returns an int error (and doesn't set errno) but can also
call Linux fallocate() which returns -1 and sets errno.

Standardize on the -1,errno convention.

Reported by Jones <[email protected]> who provided the
initial patch. This patch tested and confirmed working
by him as well.

https://bugzilla.samba.org/show_bug.cgi?id=10982

Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: David Disseldorp <[email protected]>
Fix vfs_allocate_file_space(), vfs_slow_fallocate(),
vfs_fill_sparse() to follow the -1,errno convention
for errors.

Standardize on the -1,errno convention.

Reported by Jones <[email protected]> who provided the
initial patch. This patch tested and confirmed working
by him as well.

https://bugzilla.samba.org/show_bug.cgi?id=10982

Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: David Disseldorp <[email protected]>
…tion.

Fix up the ceph, fruit, time_audit and streams_xattr modules to follow
the -1,errno convention for errors.

Reported by Jones <[email protected]> who provided the
initial patch. This patch tested and confirmed working
by him as well.

Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: David Disseldorp <[email protected]>

Autobuild-User(master): Jeremy Allison <[email protected]>
Autobuild-Date(master): Mon Dec  8 02:59:43 CET 2014 on sn-devel-104
Change-Id: Ic3a117f74e8a67c45584fdb520d92e4f0cb01c5e
Signed-off-by: Kamen Mazdrashki <[email protected]>
Reviewed-by: Andrew Bartlett <[email protected]>
A bit more specific for the caller to "know" that env key is missing

Change-Id: I4d4c2121af868d79f46f865f420336222bc67347
Signed-off-by: Kamen Mazdrashki <[email protected]>
Reviewed-by: Jelmer Vernooij <[email protected]>

Autobuild-User(master): Kamen Mazdrashki <[email protected]>
Autobuild-Date(master): Mon Dec  8 05:27:34 CET 2014 on sn-devel-104
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10965

Newer glibc versions use and unsinged integer for the count instead of
an integer.

Signed-off-by: Andreas Schneider <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
Signed-off-by: Andreas Schneider <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
Signed-off-by: Andreas Schneider <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>

Autobuild-User(master): Stefan Metzmacher <[email protected]>
Autobuild-Date(master): Mon Dec  8 16:54:51 CET 2014 on sn-devel-104
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
… path renames.

interface leases_db
{
        typedef [public] struct {
                GUID client_guid;
                smb2_lease_key lease_key;
        } leases_db_key;

        typedef [public] struct {
                file_id id;
                [string,charset(UTF8)] char *servicepath;
                [string,charset(UTF8)] char *base_name;
                [string,charset(UTF8)] char *stream_name;
        } leases_db_file;

        typedef [public] struct {
                uint32 num_files;
                [size_is(num_files)] leases_db_file files[];
        } leases_db_value;
}

As designed by metze.

Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
Will be used by lease db parsers.

Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
…path renames.

interface leases_db
{
        typedef [public] struct {
                GUID client_guid;
                smb2_lease_key lease_key;
        } leases_db_key;

        typedef [public] struct {
                file_id id;
                [string,charset(UTF8)] char *servicepath;
                [string,charset(UTF8)] char *base_name;
                [string,charset(UTF8)] char *stream_name;
        } leases_db_file;

        typedef [public] struct {
                uint32 num_files;
                [size_is(num_files)] leases_db_file files[];
        } leases_db_value;
}

As designed by metze.

Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>

Autobuild-User(master): Jeremy Allison <[email protected]>
Autobuild-Date(master): Tue Dec  9 03:44:04 CET 2014 on sn-devel-104
Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
With the notify code I've hit another case where self-sends caused
a problem.  This time messages were lost because we tried to do
multiple dispatch_rec calls from within a single inotify callback.
Only the first one was being taken care of, the others did not find
receivers.

This patch makes self-sends go through the kernel as well, the
kernel queues everything nicely for us. With dgram messaging this
should be pretty fast. If it turns out to be a performance problem,
we can solve it later by doing proper queueing in user space. We
need to completely decouple any processing from callbacks.

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
This makes iov_buf independent of talloc

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Now that iov_buf does not pull in talloc we can use it

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Notifyd should be as independent of Samba infrastructure as possible,
and it will call notify_inotify

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Early returns make code simpler

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
This gets rid of one NT_STATUS_HAVE_NO_MEMORY with its implicit return; :-)

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
tallocing first avoids having to call inotify_rm_watch

This even fixes a real error: We share inotifies between different instances,
so the rm_watch in the error paths destroys other legitimate watches

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Another case of an early return

Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Change-Id: I4a009bb7ed58ab857ac74a235bb5f580911f0d92
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: Iffd42b5b2fe5aef7d7381c91a6c635436b5e2e99
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: I4712a2a2163a57fde037511afcc1cb7bee05f12e
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: Iebf78244eaca51351d09e973fdd079f167c97ba1
Signed-off-by: Kamen Mazdrashki <[email protected]>
…ibute()

At the moment no flags are set and it works fine, since this function
is solely used in samldb during ADD requests handling.
Pre-setting a flag make it usefull for other modules and request
handlers too

Change-Id: I7e43dcbe2a8f34e3b0ec16ae2db80ef436df8bfe
Signed-off-by: Kamen Mazdrashki <[email protected]>
At the moment it works for objects with objectClass user + a common
case of removing isRecycled attribute

Change-Id: I70b0ef0ef65c13d3def82ca53ace52a85a078a37
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: Ic56c6945528b7f60becc4f0b318429f4c22c3d2e
Signed-off-by: Kamen Mazdrashki <[email protected]>
…bstone reanimate

tombstone_reanimate.c module is going to restore those attributes
and it needs a way to propagate them to DB

Change-Id: I36f30b33fa204fd28329eab01044a125f7a3f08e
Signed-off-by: Kamen Mazdrashki <[email protected]>
Before committing changes, object is still deleted - isDeleted = true

Change-Id: Ie1ab53dc594d1bfaf5b9e06316e7a1fc0dd4b8cb
Signed-off-by: Kamen Mazdrashki <[email protected]>
…equence

This way it is more visible that we work on 'deleted object' during modify
and also will help us to handle 'stop rename for deletec objects'
propertly in future

[MS-ADTS]: 3.1.1.5.3.7.3 Undelete Processing Specifics

Change-Id: I9bb644e099a4a2afcb261ad22515c9c4ce4875bb
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: I73f8cb147d6dd528428588a520826d1318a72d9f
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: Ic400c9877455e2a252c696275cab99429e1aef05
Signed-off-by: Kamen Mazdrashki <[email protected]>
…s hierarchy

Change-Id: I61be257568a778de3c4dc3d18d895d81283a0531
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: Iea92924ff6b33fa3723b104d5dfff1ce5a7a09b0
Signed-off-by: Kamen Mazdrashki <[email protected]>
At the moment tests "organizationalUnit" and "container" objects

Change-Id: I1d39b97064b2b3cda22313c521e7573b14888eca
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: I5edc6e017b576791c1575f71a625c49ccc88fe8f
Signed-off-by: Kamen Mazdrashki <[email protected]>
…_add

Change-Id: Ib88a45cea64fb661a41ca3b4a3df9dabf509fc6c
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: Ie33d92bd308262d9bfda553d6d5e2cfd98f6d7b3
Signed-off-by: Kamen Mazdrashki <[email protected]>
…_reanimation module

So we have them all in one place.

While moving, I have:
* inherited from the base class for Tombstone reanimations
* replace self.ldb with self.samdb

Change-Id: Id3e4f02cc2e0877d736da812c14c91e2311203d2
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: I7b46992c80178d40a0531b5afd71a7783068a9dd
Signed-off-by: Kamen Mazdrashki <[email protected]>
…ed objects

Now we have a module to handle to handle Tombstone reanimation
and it is better we do all the check here as usual

Change-Id: Ia5d28d64e99f7a961cfe8b9aa7cc96e4ca56192e
Signed-off-by: Kamen Mazdrashki <[email protected]>
restoring deleted object

Change-Id: Iead460d24058b160b46cf3ddedaf4d84b844da4d
Signed-off-by: Kamen Mazdrashki <[email protected]>
…C test

This way we ensure that samdb is clean before we make the test

Change-Id: I3c6fc94763807394e52b6df41548e9aba8b452c1
Signed-off-by: Kamen Mazdrashki <[email protected]>
Windows behavior in case of renaming Deleted object is:
* return ERR_NO_SUCH_OBJECT in case client is not providing
  SHOW_DELETED control
* ERR_UNWILLING_TO_PERFORM otherwise

Renaming of Deleted objects is allowed only through special
Tombstone reanimation modify request

Change-Id: I1eb33fc294a5de44917f6037988ea6362e6e21fc
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: I266b58ced814cf7ea3616862506df5b55f4f1d8c
Signed-off-by: Kamen Mazdrashki <[email protected]>
Change-Id: I87b5cc4e2b26b04d99de100a91c83cbe74e3c218
Signed-off-by: Kamen Mazdrashki <[email protected]>
… settings

No need to do anything fancy - we are going to provide user/pass anyway

TODO: Why do we need this? --abartlet

Signed-off-by: Kamen Mazdrashki <[email protected]>
TODO: I don't like the $USERNAME becoming $DC_USERNAME in the test.  This will just cause confusion --abartlet

Change-Id: I84ff628496bfa3e0538011400328585d080f21b8
Signed-off-by: Kamen Mazdrashki <[email protected]>
@kamenim kamenim force-pushed the abartlet-tombstone-reanimation-wip branch from 4cfeac5 to 11abf88 Compare December 22, 2014 01:52
abartlet pushed a commit to abartlet/samba-old that referenced this pull request Jun 17, 2015
This was reported by Pavel Březina <[email protected]>:

    We found a crash in SSSD when a tevent signal is freed in its handler, tevent
    than crashes when it access siginfo.

    sig_info is freed in signal destructor:

    > #ifdef SA_SIGINFO
    >         if (se->sa_flags & SA_SIGINFO) {
    >             if (sig_state->sig_info[se->signum]) {
    >                 talloc_free(sig_state->sig_info[se->signum]);
    >                 sig_state->sig_info[se->signum] = NULL;
    >             }
    >         }
    > #endif

    (gdb) bt
    #0  0x00007f5d4d86cc74 in tevent_signal_destructor (se=0x7f5d5370f920) at
    ../tevent_signal.c:213
    kamenim#1  0x00007f5d4d65f233 in _talloc_free_internal () from /lib64/libtalloc.so.2
    kamenim#2  0x00007f5d4d6593a3 in _talloc_free () from /lib64/libtalloc.so.2
    kamenim#3  0x00007f5d4342f3d4 in proxy_child_init_done (subreq=0x7f5d5370f600) at
    src/providers/proxy/proxy_auth.c:436
    kamenim#4  0x00007f5d4d86b0c2 in _tevent_req_error (req=req@entry=0x7f5d5370f600,
    error=error@entry=5, location=location@entry=0x7f5d43433010
    "src/providers/proxy/proxy_auth.c:356")
        at ../tevent_req.c:167
    kamenim#5  0x00007f5d4342ef5e in pc_init_sig_handler (ev=<optimized out>,
    sige=<optimized out>, signum=<optimized out>, count=<optimized out>,
    __siginfo=<optimized out>, pvt=<optimized out>)
        at src/providers/proxy/proxy_auth.c:356
    kamenim#6  0x00007f5d4d86d48c in tevent_common_check_signal (ev=0x7f5d536de670) at
    ../tevent_signal.c:428
    #7  0x00007f5d4d86f28c in epoll_event_loop (tvalp=0x7fff7b568490,
    epoll_ev=0x7f5d536de8b0) at ../tevent_epoll.c:647
    #8  epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at
    ../tevent_epoll.c:926
    #9  0x00007f5d4d86d7d7 in std_event_loop_once (ev=0x7f5d536de670,
    location=0x7f5d50faedc3 "src/util/server.c:668") at ../tevent_standard.c:114
    #10 0x00007f5d4d869fbd in _tevent_loop_once (ev=ev@entry=0x7f5d536de670,
    location=location@entry=0x7f5d50faedc3 "src/util/server.c:668") at
    ../tevent.c:530
    #11 0x00007f5d4d86a15b in tevent_common_loop_wait (ev=0x7f5d536de670,
    location=0x7f5d50faedc3 "src/util/server.c:668") at ../tevent.c:634
    #12 0x00007f5d4d86d777 in std_event_loop_wait (ev=0x7f5d536de670,
    location=0x7f5d50faedc3 "src/util/server.c:668") at ../tevent_standard.c:140
    #13 0x00007f5d50f96863 in server_loop (main_ctx=0x7f5d536dfac0) at
    src/util/server.c:668
    #14 0x00007f5d5180aa42 in main (argc=8, argv=<optimized out>) at
    src/providers/data_provider_be.c:2909

    But then it is accessed again in tevent_common_check_signal:

    > #ifdef SA_SIGINFO
    >         if (clear_processed_siginfo) {
    >             uint32_t j;
    >             for (j=0;j<count;j++) {
    >                 uint32_t ofs = (counter.seen + j)
    >                     % TEVENT_SA_INFO_QUEUE_COUNT;
    >                 memset((void*)&sig_state->sig_info[i][ofs],
    >                     '\0',
    >                     sizeof(siginfo_t));
    >             }
    >         }
    > #endif

    (gdb) bt
    #0  0x00007fd7ba400505 in memset (__len=<optimized out>, __ch=<optimized out>,
    __dest=<optimized out>) at /usr/include/bits/string3.h:84
    kamenim#1  tevent_common_check_signal (ev=0x7fd7bfddf670) at ../tevent_signal.c:459
    kamenim#2  0x00007fd7ba40228c in epoll_event_loop (tvalp=0x7fff85536430,
    epoll_ev=0x7fd7bfddf8b0) at ../tevent_epoll.c:647
    kamenim#3  epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at
    ../tevent_epoll.c:926
    kamenim#4  0x00007fd7ba4007d7 in std_event_loop_once (ev=0x7fd7bfddf670,
    location=0x7fd7bdb417c3 "src/util/server.c:668") at ../tevent_standard.c:114
    kamenim#5  0x00007fd7ba3fcfbd in _tevent_loop_once (ev=ev@entry=0x7fd7bfddf670,
    location=location@entry=0x7fd7bdb417c3 "src/util/server.c:668") at
    ../tevent.c:530
    kamenim#6  0x00007fd7ba3fd15b in tevent_common_loop_wait (ev=0x7fd7bfddf670,
    location=0x7fd7bdb417c3 "src/util/server.c:668") at ../tevent.c:634
    #7  0x00007fd7ba400777 in std_event_loop_wait (ev=0x7fd7bfddf670,
    location=0x7fd7bdb417c3 "src/util/server.c:668") at ../tevent_standard.c:140
    #8  0x00007fd7bdb29343 in server_loop (main_ctx=0x7fd7bfde0ac0) at
    src/util/server.c:668
    #9  0x00007fd7be39ca42 in main (argc=8, argv=<optimized out>) at
    src/providers/data_provider_be.c:2909

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11308

Signed-off-by: Stefan Metzmacher <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>

Autobuild-User(master): Jeremy Allison <[email protected]>
Autobuild-Date(master): Tue Jun  2 21:02:11 CEST 2015 on sn-devel-104
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants